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Abstract:  The  RASR  team  developed  a  system  for  the  coordination  of  groups  of  unmanned  ground 
vehicles  (UGVs)  that  can  execute  a  variety  of  military  relevant  missions  in  dynamic  urban 
environments.  Historically,  UGV  operations  have  been  primarily  performed  via  tele-operation, 
requiring  at  least  one  dedicated  operator  per  robot,  and  requiring  substantial  real-time  bandwidth  to 
accomplish  those  missions.  Our  team  goal  was  to  develop  a  system  that  can  provide  long  term  value  to 
the  war-fighter,  utilizing  MAGIC  2010  as  a  stepping  stone.  To  that  end,  we  self-imposed  a  set  of 
constraints  that  would  force  us  to  develop  technology  that  could  readily  be  used  by  the  military  in  the 
near  term:  (1)  Use  a  relevant  (deployed)  platform,  (2)Use  low-cost,  reliable  sensors,  (3)  Develop  an 
expandable  and  modular  control  system  with  innovative  software  algorithms  to  minimize  the  computing 
footprint  required,  (4)  Minimize  required  communications  bandwidth  and  handle  communication  losses 
and,  (5)  Minimize  additional  power  requirements  to  maximize  battery  life  and  mission  duration 


Introduction:  Small  unmanned  ground  vehicles  (UGVs)  save  lives  in  Iraq  and  Afghanistan  by 
distancing  humans  from  dangerous  areas.  These  robots  are  instrumental  in  EOD  applications,  including 
improvised  explosive  device  (IED)  detection  and  neutralization.  However,  current  controllers  require  at 
least  one  operator  for  each  robot.  An  operator  must  tele-operates  a  single  UGV  to  the  suspect  object 
where  the  operator  remotely  manipulates  and  deactivates  the  IED.  All  of  these  operations  are  performed 
using  remote  video  requiring  the  complete  attention  of  the  operator. 


Experiment:  To  help  break  the  1:1  ratio  needed  for  unmanned  vehicles  operation  and  to  promote 
autonomous  control  of  small  Unmanned  Ground  Vehicles,  the  Defence  Science  &  Technology 
Organisation  (DSTO)  in  Australia  and  the  Research  Development  &  Engineering  Command 
(RDECOM)  in  USA  took  the  lead  in  organizing  MAGIC  2010.  This  challenge  requires  multi-vehicle 
robotic  teams  that  can  execute  an  intelligence,  surveillance  and  reconnaissance  mission  in  a  dynamic 
urban  environment.  To  complete  the  challenge,  competitors  must:  (i)  accurately  and  completely  explore 
and  map  the  challenge  area;  (ii)  correctly  locate,  classify  and  recognize  all  simulated  threats;  and  (iii) 
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complete  all  phases  within  3.5  hours. 

The  challenge  event  was  conducted  in  Australia  during  November  2010. 

The  challenge  is  mindful  that,  at  the  current  state  of  autonomy,  operators  will  still  provide  oversight  of 
the  UGVs.  Therefore,  this  challenge  also  forces  developers  to  design  a  Human  Machine  Interface 
(HMI)  that  minimizes  operator  workload  and  increase  overall  effectiveness. 

The  RASR  team  believes  that  the  challenge  is  a  realistic  step  toward  the  next  generation  of  small  UGV 
operation  and  that  our  solution  introduces  a  new  philosophy  of  distributed  control  that  is  well  suited  for 
the  platform  size  and  reduced  communication  bandwidth. 

Results  and  Discussion:  The  RASR  Team  (Reconnaissance  and  Autonomy  for  Small  Robots)  was 
selected  to  be  among  the  6  finalists  for  MAGIC  2010.  The  RASR  Team  placed  third  in  the  final 
competition.  The  technology  developed  by  Robotic  Research  will  be  further  advanced  in  order  to  be 
transitioned  to  small  unmanned  ground  vehicles  for  US  military. 
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The  Reconnaissance  and  Autonomy  for  Small  Robots  (RASR): 
MAGIC  2010  Challenge 

Alberto  Lacaze  and  Karl  Murphy ,  Robotic  Research ,  LLC 
Mark  Del  Giorno ,  Del  Services ,  LLC 


ABSTRACT 


The  RASR  team  developed  a  system  for  the  coordination  of  groups  of  unmanned  ground  vehicles 
(UGVs)  that  can  execute  a  variety  of  military  relevant  missions  in  dynamic  urban  environments. 
Historically,  UGV  operations  have  been  primarily  performed  via  tele-operation,  requiring  at  least  one 
dedicated  operator  per  robot,  and  requiring  substantial  real-time  bandwidth  to  accomplish  those 
missions.  Our  team  goal  was  to  develop  a  system  that  can  provide  long  term  value  to  the  war-fighter, 
utilizing  MAGIC  2010  as  a  stepping  stone.  To  that  end,  we  self-imposed  a  set  of  constraints  that  would 
force  us  to  develop  technology  that  could  readily  be  used  by  the  military  in  the  near  term: 

•  Use  a  relevant  (deployed)  platform 

•  Use  low-cost,  reliable  sensors 

•  Develop  an  expandable  and  modular  control  system  with  innovative  software  algorithms  to 
minimize  the  computing  footprint  required 

•  Minimize  required  communications  bandwidth  and  handle  communication  losses 

•  Minimize  additional  power  requirements  to  maximize  battery  life  and  mission  duration 


1.  Introduction 

Small  unmanned  ground  vehicles  (UGVs)  save 
lives  in  Iraq  and  Afghanistan  by  distancing 
humans  from  dangerous  areas.  These  robots  are 
instrumental  in  EOD  applications,  including 
improvised  explosive  device  (IED)  detection 
and  neutralization.  However,  current 
controllers  require  at  least  one  operator  for  each 
robot.  An  operator  must  tele-operates  a  single 
UGV  to  the  suspect  object  where  the  operator 
remotely  manipulates  and  deactivates  the  IED. 
All  of  these  operations  are  performed  using 


remote  video  requiring  the  complete  attention 
of  the  operator. 

To  help  break  this  1:1  ratio  and  to  promote 
autonomous  control  of  small  Unmanned 
Ground  Vehicles,  the  Defence  Science  & 
Technology  Organisation  (DSTO)  in  Australia 
and  the  Research  Development  &  Engineering 
Command  (RDECOM)  in  USA  took  the  lead  in 
organizing  MAGIC  2010.  This  challenge 
requires  multi-vehicle  robotic  teams  that  can 
execute  an  intelligence,  surveillance  and 
reconnaissance  mission  in  a  dynamic  urban 
environment.  To  complete  the  challenge 
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competitors  must:  (i)  accurately  and  completely 
explore  and  map  the  challenge  area;  (ii) 
correctly  locate,  classify  and  recognize  all 
simulated  threats;  and  (iii)  complete  all  phases 
within  3.5  hours.  The  challenge  event  will  be 
conducted  in  Australia  during  November  2010. 

The  challenge  is  mindful  that,  at  the  current 
state  of  autonomy,  operators  will  still  provide 
oversight  of  the  UGVs.  Therefore,  this 
challenge  also  forces  developers  to  design  a 
Human  Machine  Interface  (HMI)  that 
minimizes  operator  workload  and  increase 
overall  effectiveness. 

The  RASR  team  believes  that  the  challenge  is  a 
realistic  step  toward  the  next  generation  of 
small  UGV  operation  and  that  our  solution 
introduces  a  new  philosophy  of  distributed 
control  that  is  well  suited  for  the  platform  size 
and  reduced  communication  bandwidth.  See 
Figure  1. 


1.1  Design  Philosophy 

Our  long  term  goal,  resulting  from  our 
competing  in  Magic  2010,  is  to  develop 
relevant  technology  for  control  and 
coordination  of  multiple  small  UGVs.  Our 
choices  of  platform,  sensing,  and  computational 
engine  had  to  provide  a  clear  pathway  to 
deployment.  Decisions  taken  throughout  the 
design  process  have  been  guided  to  provide  a 


low  cost  autonomous  mission  module  that  is 
deployable.  Cost  was  a  large  parameter  in  our 
sensor,  localization  and  radio  selections.  We 
could  have  “bought  our  way  out  of  the 
problem”  in  many  circumstances  by  purchasing 
more  LADAR  sensors,  a  higher  quality 
navigation  components,  or  advanced  computing 
hardware,  but  those  decisions  would  have 
yielded  a  system  that  would  be  prohibitively 
expensive  for  broad  acceptance. 

1.1.1  Risk  reduction 

Our  overall  approach  was  based  on  an  up-front 
risk  reduction  process.  That  process  flowed  to 
both  software  and  hardware  requirements 
during  the  initial  design  phase.  High-risk 
elements  were  identified  and  considered  as  key 
factors  when  performing  trade  studies  and 
choosing  among  various  approaches  to  solving 
the  MAGIC  problem. 

Hardware  risks  were  often  mitigated  using  as 
much  proven  COTS  sensors  and  components  as 
possible.  When  custom  hardware  designs  were 
required,  they  were  fabricated  early  in  the 
process  to  allow  time  for  the  “gotchas”  that 
often  follow  with  custom  hardware.  As  an 
example:  our  custom  navigation  electronics 
solution  running  on  the  Talon  platform  is 
currently  at  revision  level  8.  Some  revisions 
were  based  on  required  redesign  efforts,  and 
others  were  based  on  additional  requirements 
discovered  as  the  design  process  evolved.  By 
identifying  that  component  as  a  high-risk 
element  early  in  the  process  and  tackling  the 
design  early,  each  of  those  intermediate 
iterations  were  much  lower  risk,  making  for  a 
better  product  in  the  end. 

Software  risks  were  also  addressed  by  working 
on  the  hard  problems  first.  By  the  time  of  our 
site  visit,  we  had  already  coded  and  tested 
substantial  portions  of  the  identified  high-risk 
elements  (coordination  planners,  real-time 
control  &  navigation  of  a  tracked  system, 
operations  in  comms-denied  environments), 
while  lower-risk  items  were  delayed  since  they 
didn’t  have  as  much  potential  of  changing  the 
overall  system  architecture. 
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1.1.2  A  Relevant  Platform 

The  Intelligent  Systems  Division  at  NIST  tests 
small  robotic  platforms  utilizing  a  calibrated 
ASTM  qualified  course.  Vehicles  that  are 
currently  in  theatre  or  are  being  considered  for 
deployment  are  tested  in  this  “do  or  die”  test. 
Tested  vehicles  in  the  MAGIC  weight  category 
include:  The  Dragon  Runner,  Souryu  IV, 
G2bot,  Element,  UMRS  2009,  Kenaf,  Quince, 
Matilda  1,  Matilda  2,  Packbot  510,  Mutech-R4, 
Helios,  KOHGA,  Talon,  TeleMax,  Caliber 
MK3,  Andros  HD-1  J  and  Andros  Mini. 

These  vehicles  have  one  common  denominator: 
they  are  all  tracked.  The  DoD  understands  that 
wheeled  vehicles  in  this  weight  class  are  simply 
not  relevant  for  current  operations.  It  was  our 
decision  from  the  beginning  of  the  program  to 
utilize  a  tracked  vehicle  even  though  wheeled 
vehicles  would  simplify  the  navigation  and 
mapping  aspects  of  this  competition.  We 
selected  QinetiQ-NA’s  (QNA)  Talon  platform, 
however,  we  want  to  emphasize  that  the 
resulting  solution  can  be  adjusted/adapted  to 
smaller  platforms,  like  the  Dragon  Runner.  See 
Figure  2. 

1.1.3  Sensing 


sufficient  to  provide  the  localization  accuracy 
needed  to  accurately  map  the  scenario. 

There  are  a  variety  of  COTS  navigation 
solutions  in  the  US  $50- 100k  range  that  would 
provide  inertial  backing  to  support  the  accuracy 
requirements.  It  is  our  opinion,  that  the  overall 
cost  of  an  autonomous  system  that  has  a 
component  in  this  cost  range  would  not  be 
appealing  for  military  or  other  applications. 
Therefore,  we  developed  for  MAGIC  a 
navigation  unit  in  the  US  $10k  range  and 
tailored  it  to  the  mission  constraints. 

1.1.5  Data  Radios 

In  current  teleoperated  systems,  where  a 
constant  communication  links  is  required,  the 
DoD  customers  usually  opt  for  radios  that  are 
not  at  the  top  of  the  price  range.  This  is  evident 
by  the  radio  selection  in  the  Talon  systems  and 
in  iRobot’s  Packbot,  arguably  the  most  popular 
platforms  currently  in  theater.  Therefore,  we 
opted  for  radios  in  the  same  price  range, 
concentrating  instead  on  the  autonomy  and 
software  smarts  to  deal  with  comms  losses 
which  are  bound  to  happen  —  no  matter  how 
expensive  the  radio.. 

1.1.6  Computing  platform 


Sensors  are  often  the  culprits  of  cost  escalation 
for  autonomous  systems.  To  keep  the  cost 
down,  we  utilized  a  single  COTS  LADAR 
sensor.  Utilizing  multiple  LADARs  would 
have  made  the  problem  simpler,  and  reduced 
the  software  requirements  at  the  cost  of 
reducing  the  deployability  of  the  system.  We 
chose  to  take  the  harder  path  and  make  up  for  it 
in  software.  Although  this  approach  added  risk, 
these  risks  were  identified  and  mitigated  early- 
on  in  our  design  process  for  an  overall 
reduction  in  our  risk  assessment  process. 

1.1.4  Navigation  System 

IMUs  are  another  example  of  where  we  could 
have  “bought  our  way  out  of  the  problem.” 

In  urban  warfare,  it  is  common  to  encounter 
GPS  denied  and  multipath  situations  where 
GPS  (even  subscription  versions)  is  not 


The  MAGIC  competition  requires  advanced 
multi-asset  coordination  and  planning  systems 
in  order  to  accomplish  the  mission.  There  is  a 
great  temptation  to  “throw  computers  at  the 
problem.”  Our  MAGIC  team  resisted,  opting 


Figure  2  Eight  RASR-Bots  map  the  interior  of  a 
large  building 
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for  designing  algorithms  that  execute  within  a 
high-efficiency  software  architecture 
framework. 

We  selected  a  single  commercial  computing 
platform,  a  Mac  Mini.  Although  not  rugged  for 
field  deployment,  several  ruggedized  platforms 
have  similar  computational  power.  We 
implemented  a  mixture  of  real-time  control 
algorithms  with  high-  and  mid-level  planners 
that  work  as  a  unit  to  use  as  little  computing 
power  as  needed.  In  addition  to  lower  cost  and 
complexity  another  advantage  of  minimizing 
computing  hardware  is  to  lessen  the  burden  on 
the  battery  system,  allowing  for  missions  of  up 
to  4  hours  without  recharging  the  Talon. 

1.1.7  Relevant  autonomous  mobility  and 
coordination  software 

Since  much  of  our  design  criteria  heavily 
weights  the  overall  cost  and  deployability  of  the 
team,  the  autonomous  mobility  system  must  be 
robust  enough  to  cope  with  this  decision.  In 
particular  it  must  be  able  to  cope  with  the 
treaded  platform,  the  midrange  sensors 


capabilities  and  inexpensive  IMU  components. 
Moreover,  the  vehicle  cooperation 
infrastmcture  must  provide  intelligent  behavior 
while  the  vehicle  is  out  of  comms.  We  spent  a 
significant  portion  of  our  budget  and  time 
designing  and  implementing  software  for  the 
hard  constrains  that  the  MAGIC  2010  imposed 
on  us  as  well  as  our  own  constraints  from  what 
our  team  partners  at  GDRS  and  QNA  have 
experienced  from  deployed  systems. 

1.2  Overall  system  description 

Hardware  and  software  solutions  are 
summarized  in  this  section  and  are  explained  in 
detail  in  the  following  sections. 

1.2.1  Hardware 

The  RASR  Team  is  composed  of  eight 
platforms  (6  sensor  UGVs  and  2  dismptor 
UGVs),  and  2  Operator  Control  Units  (OCUs). 
Each  robot  has  a  sensor  pod  providing  360  by 
90  degree  LADAR  coverage,  360  by  90  degree 
camera  coverage,  an  INU,  two  data  radios  (data 
and  E-STOP),  and  a  main  computer. 

1.2.2  Systems  architecture 

The  systems  architecture  is  composed  of  a 
distributed/hierarchical  control  architecture  and 
a  matching  Human  Machine  Interface.  See 
Figure  3.  The  control  architecture  is  composed 
of  the  Coordination  Layer,  the  Autonomous 
Mobility  Layer  and  the  Vehicle  Platform  Layer. 
Each  layer  in  the  control  hierarchy  contains 
Sensing, 

Modeling  and  Planning  (S,M,P)  modules.  The 
Coordination  Layer  maintains  the  overall 
situational  awareness  and  exposure  measures. 
This  layer  is  distributed  among  the  robots  and 
the  base  station.  At  the  OCU,  a  corresponding 
Coordination  Human  Machine  Interface 
(CHMI)  provides  the  operators  with 
coordination  oversight.  The  Autonomous 
Mobility  Layer  performs  local  path  planning 
and  OOI  neutralization.  A  separate  copy 
resides  on  each  robot  At  the  OCU,  a 
corresponding  Autonomy  HMI  (AHMI)  and  a 
Neutralization  HMI  (NHMI)  assign  coarse 
scheduling  tasks  to  provide  oversight  for  these 
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operations.  The  Vehicle  Platform  Layer 
provides  low  level  control  functions  including 
path  following,  communication  infrastructure 
and  e-stop  functions.  At  the  OCU,  the  Platform 
HMI  (PHMI)  provides  the  operator  with 
oversight  of  platform  issues.  The  different 
functionalities  of  the  HMI  have  been  integrated 
into  a  single  interface. 

1.3  Ground  Vehicle  Component  & 
Systems 

Selection  of  a  military  relevant  platform  was  a 
key  decision  that  drove  many  aspects  of  our 
MAGIC  2010  entry.  Based  on  currently 
deployed  platforms,  this  meant  the  use  of  a 
tracked,  skid-steer  vehicle.  The  navigation 
would  be  more  difficult  than  on  a  wheeled 


platform  that  can  more  heavily  rely  on  wheel 
encoders.  See  Figure  4. 

Terrain  sensors  added  to  the  base  platform 
includes  a  COTS  LADAR  configured  in  an 
innovative  pattern  and  three  fish  eye  cameras. 
The  cameras  are  used  for  Object  Of  Interest 
(OOI)  detection  and  tracking,  visual  odometry, 
and  teleoperation  when  required.  A  custom 
made  INU  with  GPS  supply  the  core  navigation 
solution.  Two  different  radios  provide  a  data 
link  and  a  remote  E-stop  capability.  A  Core  II 
duo  processing  board  hosts  the  control  system. 
A  power  distribution  system  allows  hot 
swapping  of  batteries.. 


360  deg  LADAR  coverage 


360  deg  field  of  view  video 


Dual  Core  2.6GHz 
processor 


Proven  skid 
steer  platform 


Accurate  inertial  with  GPS 


38kg  as  shown 


Custom 
E-stop  rad 


Power  distribution  /  E-stop  board 
with  battery  hot  swap  capability 
(4  hour  endurance) 


COTS  game  controller  COTS  802. 1 1  radio 


Figure  4  RASR-Bot  detail 
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2.  UVS  Autonomy  and 
Coordination  Strategy 

2.1  Introduction 

MAGIC  2010  introduces  a  number  of  real 

world  planning  challenges: 

•  Coordinating  groups  of  vehicles  is  a  highly 
dimensional  search  problem  that  cannot  be 
fully  expanded  using  realistic 
computational  capabilities. 

•  Uncertainties  of  radio  communication 
makes  centralized  approaches  vulnerable  to 
outages. 

•  Algorithmic  cost  spaces  that  include  the 
opposing  requirements  of  searching  and 
neutralizing,  create  unbalanced  search 
heuristics. 

•  Autonomous  mobility  in  a  small  platform 


has  stringent  weight,  power,  and  size 
constraints. 

•  A  hybrid  set  of  system  capabilities  from 
both  a  mission  standpoint  (sensor  UGVs  vs. 
disruptor  UGVs),  and  from  a  computational 
standpoint  (UGVs  vs.  OCUs). 

•  The  need  to  perform  localization  and 
mapping  indoors,  without  GPS. 

The  RASR  approach  unravels  these  challenges 
to  provide  a  computationally  feasible 
coordination  and  autonomous  system  that  is 
highly  resilient  to  communications  and  GPS 
outages. 

2.2  Overall  Planning  and 

Coordination  System 

The  planning  and  coordination  system  is 
hierarchically  organized  providing  a  distributed 
coordination  layer  and  a  group  of  specialized 
planners  to  solve  the  mapping  and 
neutralization  problems.  Figure  5  shows  the 
overall  system  diagram. 

The  system  is  organized  so  the  elements  at  the 
top  of  the  hierarchy  are  coarse,  with  slower 
planning  cycles,  while  at  the  bottom  of  the 
hierarchy,  the  cycles  are  fast,  and  the  resolution 
is  high  within  a  reduced  scope.  At  the  top  of 
the  hierarchy,  the  system  has  a  coordination 
layer  that  plans  the  synchronized  motion  of  the 
UGV  group.  It  performs  task  allocation  and 
rough  scheduling  of  the  group.  This  layer 
resides  on  each  UGV  as  well  as  on  the  Operator 
Control  Units.  Each  module  in  the  layer  is 
composed  of  a  Coordination  Planner  (CP) 
which  interacts  with  the  Global  Autonomous 
mobility  Model  (GAM)  and  the  Global  Mission 
Model  (GMM).  The  OCU  also  contains  a 
Situational  Awareness  Model  (SAM).  The 
Coordination  Layer  uses  radio  communications 
to  maintain  database  coherency  and  to 
propagate  plans. 

Each  vehicle  has  an  Autonomous  Mobility 
Layer  (AM).  AM  is  composed  of  a  local 
version  of  a  layered  map  and  exposure 
database,  the  Local  Autonomous  mobility 
Model  (LAM)  and  the  Local  Mission  Model 
(LMM).  The  planner  at  this  level  solves  the 
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problem  of  single  vehicle  navigation  and  local 
coordination  in  the  case  of  Neutralization.  This 
layer  receives  coarse  plans  from  the 
Coordination  Layer,  and  provides  plans  that 
minimize  local  exposure  and  optimize  mobility 
constraints  for  the  UGV.  These  plans  are  sent 
to  the  Vehicle  Platform  Layer  where  the  task  is 
to  transform  these  plans  into  actuator 
commands.  The  Vehicle  Platform  Layer 
maintains  the  navigation  solution  and  provides 
the  E-stop  Controller  (EC). 

2.3  Representation 

The  system  provides  three  concurrent 
representations: 

The  Autonomous  Mobility  World  Model 
provides  traversability  information  that  is  used 
by  the  different  levels  to  calculate  the  costs  of 
the  plans  from  a  mobility  standpoint. 

Mission  Specific  World  Model  provides 
information  about  the  Object(s)  of  Interest 
(OOIs),  and  their  motion  prediction.  It  is 
represented  in  the  form  of  a  probability  density 
function  of  each  OOI  being  present  at  a  location 
at  a  particular  moment  in  time.  The  system  will 
also  maintain  the  exposure  to  OOIs  (mobile  or 
static)  based  on  the  mapped  areas.  A 
probability  of  detonation  is  computed  using 
both  layers. 

Situational  Awareness  World  Model  is 
designed  for  operator  consumption.  This 
representation  will  allow  the  operator  to 
understand  the  environment,  and  intervene  if 
necessary. 

2.4  Coordination  Layer 

The  coordination  layer  resides  on  each  UGV 
and  on  each  OCU.  Our  overall  philosophy 
embeds  coordination  capabilities  on  each  robot 
in  the  architecture.  The  communications 
between  robots  is  kept  at  a  minimum  by  only 
propagating  bounds  of  the  solutions  found  in 
the  nodes  called  “contracts.”  When 
communications  connect  the  UGVs  and  the 


OCUs;  the  coordination  layer  benefits  from  the 
larger  number  of  computational  units.  In  those 
cases,  the  larger  number  crunching  capabilities 
of  some  nodes,  like  the  OCUs,  will  provide 
search  bounds  to  the  rest  of  the  robot  team. 
When  communications  are  poor  and  UGVs  are 
isolated,  they  still  can  coordinate  in  their  local 
communication  neighborhood.  It  has  been 
shown  that  this  system  is  guaranteed  to 
outperform  an  auctioning  coordination  strategy. 
The  Robotic  Research  MPAC  library  (MPAC  is 
software  and  system  developed  for  autonomy  of 
small  unmanned  surface  vehicles)  provides  the 
search  engine  in  the  Coordination  Layer 
Planner. 

2.4. 1  Generalized  Formulation  of  Area 
Search 

The  search  area  is  decomposed  into  a  number  of 
smaller  areas  called  “countries”.  Each  country 
has  a  point,  called  the  “capital”.  From  the 
capital,  the  robot  can  see  every  other  point  in 
the  country.  This  depends  on  the  range  of  the 
robot  sensors  and  the  line  of  sight  around 
known  obstacles.  The  creation  of  countries  and 
capitals  is  described  in  the  following  section. 

This  generalized  formulation  allows  other  types 
of  missions  to  be  performed  in  addition  to 
search-only  missions.  Additionally,  the  ability 
to  plan  for  multiple  types  of  vehicles  is  made 
possible  by  abstracting  the  tasks  into  the 
starting  conditions,  ending  conditions,  and 
resources  required  for  completion.  Through 
this  basic  formulation,  a  wide  variety  of 
pertinent  missions  can  be  managed  on-the-fly 
by  a  group  of  UGVs. 

2.4.2  K-Means  Line  of  sight:  KML 

A  new  algorithm  was  designed  to  compute  the 
countries  and  capitals.  The  idea  behind  KML  is 
to  find  the  smallest  number  of  points  (capitals) 
from  which  all  the  tiles  in  the  desired  search 
space  can  be  viewed  taking  under  consideration 
the  line  of  sight.  Once  this  is  accomplished,  the 
problem  of  coverage  becomes  a  travelling 
salesman  without  having  to  do  LOS  checks  or 
coverage  propagation  throughout  the  search. 
The  space  of  search  collapses  by  several 
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(hundreds  in  most  cases)  orders  of  magnitude. 

See  Figure  6.  Some  of  the  characteristics  of 

KML: 

•  In  the  family  of  K-means,  but  guarantees 
LOS 

•  Minimizes  number  of  points  with  similar 
convergence  and  warranties  as  K-means 

•  Minimizes  the  sums  of  squares.  This  is 
very  important  because  it  means  that  it 
computes  paths  that  minimize  distances  to 


Figure  6.  KML  automatically  subdivides  the 
total  area  into  smaller  areas  called 
(i  countries”,  each  with  a  “ capital”.  From  the 
capital  the  robot  can  see  the  entire  country, 
given  the  known  obstacles. 


•  Efficient  implementation 

2.4.3  Anytime  and  Memory-Constrained 
Search  Algorithms 

All  algorithms  are  designed  to  be  “anytime 
algorithms.”  These  algorithms  can  find 
solutions  quickly  (fast  response)  and  will 
continue  to  improve  solutions  when  given  more 
time  (ongoing  improvement  until  the  optimum 
solution  is  found).  This  approach  also  allows 
the  algorithms  to  quickly  respond  and  replan  as 
new  information  is  gathered,  e.g.  a  new 
blockage  is  discovered.  In  addition,  these 
algorithms  have  been  designed,  implemented, 
and  tested  to  work  within  the  memory  available 
(from  a  single  megabyte  to  multiple  gigabytes). 
For  sufficiently  complex  missions,  the 
distributed  algorithm  will  return  a  non-optimal 
plan  immediately  while  continuing  to  improve 
the  solution  in  the  background. 

Complementary  to  the  anytime/memory  aspects 
is  the  online  optimality  guarantees.  By  nature 
of  the  bounded  search,  a  set  of  upper  and  lower 
bounds  on  the  optimal  solution  are  constantly 
being  calculated.  These  bounds  guarantee  three 
important  benefits:  (1)  the  system  is  able  to 
estimate  how  close  the  current  solution  is  to  the 
optimal  solution  as  the  search  progresses,  (2) 
the  system  is  capable  of  proving  it  has  found 
the  optimal  solution,  and  (3)  the  system  is 
guaranteed  to  find  the  optimal  solution  when 
given  sufficient  computational  resources. 

2.4.4  Multi-Vehicle  Coordination 
Algorithms 

In  a  distributed  environment,  the  propagation  of 
information,  the  transfer  of  tasks,  and 
communications  topologies  have  been 
addressed.  Robotic  Research’s  “MPAC” 
system  provides  a  contract-based  multi-vehicle 
coordination  system  which  provides  an  efficient 
way  to  share  information  and  exchange  task 
responsibilities  even  under  degraded 
communications. 


areas  to  be  surveyed 
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2.5  Path  Planning  at  the  Autonomous 
Mobility  Layer 

The  autonomous  mobility  layer  is  based  on  the 
High  Maneuverability  Planner  (HMP).  This 
kino-dynamic  planner  has  been  utilized  by  U.S. 
Army  unmanned  platforms  for  a  variety  of 
programs  including:  the  Collaborative 

Technology  Alliance  (U.S.  Army  Research 
Laboratory’s  (ARL)  Robotics  Collaborative 
Technology  Alliance  (RCTA)),  Safe  Ops  (US 
Army,  TARDEC,  David  Kowachek,  PM),  and 
the  Autonomous  Navigation  System  (PM  FCS). 
The  implementation  on  MAGIC  is  the  first  time 
it  has  been  applied  to  small  robots  maneuvering 
in  tight  quarters. 

This  module  generates  trajectories  for  each 
UGV,  avoiding  obstacles  while  meeting  the 
constraints  of  the  plans  created  by  the 
coordination  layer.  See  Figure  7.  An  instance 
of  this  module  resides  on  each  UGV.  The  path 
planner's  input  is  a  3D  representation  of  its 
vicinity  in  a  relative  coordinate  frame  (LAM). 
It  outputs  a  trajectory  to  be  followed  by  the 
Vehicle  Platform  Layer  (VPL).  The  HMP 
combines  all  sensor  information  into  a  single 
representation  of  the  environment  that  is  then 
utilized  to  evaluate  the  cost  of  performing 
different  actions.  As  such,  the  environmental 
representation  includes  morphological 


Figure  7  Simulation  of  the  HMP  generating  a 
trajectory  through  a  staircase  with  rubble.  The 
HMP  can  handle  both  holonomic  and  non- 
holonomic  platforms. 


information,  as  well  as  slippage 
characterization.  The  resulting  trajectories  are 
sequences  of  vehicle  state/time  pairs  that  the 
VPL  follows.  Among  other  things,  the  state 
information  includes  the  desired  vehicle 
position  and  attitude. 

3.  Sensors,  Processing  & 
Mapping  for  UGVs 

Although  sensing  is  not  meant  to  be  the  core 
problem  of  the  MAGIC  2010  challenge,  several 
of  the  sensing  requirements  are  not  trivial.  The 
system  requires  fusing  LADAR  and  color 
vision  to  create  maps  and  to  Objects  of  Intrest. 
Robotic  Research,  Del  Services,  and  GDRS 
have  a  long  history  of  developing  and  testing 
sensing  algorithms  in  robotic  systems. 
Previous  research  allowed  us  to  identify  which 
areas  needed  work  for  the  small  UGVs.  The 
sensing  system  fuses  LADAR  and  color 
information  to  classify  terrain,  map  the  areas, 
and  autonomously  recognize  and  classify  the 
static  and  dynamic  OOIs.  See  Figure  8. 

3.1  Sensor  Processing 

Static  feature  detection  is  performed  by  fusing 
morphological  information  provided  by  the 
LADAR  together  with  color  and  texture 
information  provided  by  the  cameras.  The  most 
challenging  aspect  of  the  sensing  requirements 
is  the  detection  and  prediction  of  movers.  The 
team  has  shown  this  capability  in  many 
programs,  for  larger  platforms  (SafeOps  and 
CTA)  and  smaller  platforms  (SBIR  DARPA 
SB082-29  Multi-Sensor  Detection  and  Tracking 
using  Traversability  Based  Prediction,  Dr. 
Robert  Mandelbaum,  PM. 

3.2  Mapping  Requirements 

Three  different  models  are  maintained  for 
different  purposes:  autonomy,  mission  tasks  and 
Autonomous  mobility  Models  (AM). 

Two  maps  of  the  world  are  be  maintained  for 
mobility  purposes;  The  Global  Autonomous 
mobility  Model  (GAM)  and  the  Local 
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Figure  8  Sensor  processing  fuses  video  and 
LADAR  information  to  sense,  track  and  predict 


OOIs 


Autonomous  mobility  Model  (LAM).  LAM  is 
a  vehicle  centered  grid  based  214  D  map.  Each 
tile  in  the  map  provides  classification, 
elevation,  density,  and  the  presence  of  positive 
or  negative  obstacles. 

Features  in  the  LAM  are  used  by  the  LADAR 
registration  algorithms  to  merge  all  of  the 
individual  LAMs  into  a  single  coherent  GAM. 
The  GAM  is  displayed  on  the  OCU. 

3.3  Map  Processing 

The  OCU  has  a  model  of  the  maps  with  the 
current  best  estimate  of  the  vehicles  navigation 
solution.  As  navigation  information  becomes 
available,  including  single  robot  loop  closures 
and  multi-robot  loop  closures,  maps  are 
regenerated  from  relative  maps  solutions.  The 
resulting  maps  are  then  displayed  on  the  OCU 
and  used  for  generating  the  final  map 
submission. 


3.3.1  Mission  Model  (MM) 

Since  part  of  the  mission  is  to  correctly  identify 
and  neutralize  OOIs,  the  MM  is  tasked  with 
maintaining  and  predicting  the  knowledge 
about  the  humans.  Dynamic  OOIs  are  detected 
using  onboard  sensors  and  through  the  metadata 
provided  by  the  UAV.  Each  vehicle  tracks  and 
classifies  humans  in  its  field  of  view  and  stores 
them  in  the  Local  MM  (LMM),  the  Global  MM 
is  then  used  to  maintain  coherency  of 
classification  as  the  non-combatants  and 
referees  move  in  and  out  of  the  field  of  view  of 
the  UGVs.  The  MM  at  both  levels  also  has  the 
task  of  performing  dynamic  OOI  prediction.  In 
order  to  perform  this  prediction,  the  Terrain 
Aware  Coordination  Tool  for  Intelligent 
Control  (TACTIC)  is  utilized.  TACTIC  was 
designed  by  Robotic  Research  for  an  Army  War 
College  challenge  organized  and  funded  by 
ARL  in  2004.  RR  won  this  challenge  by 
utilizing  the  TACTIC  toolset. 

TACTIC  approximates  the  results  of  a  Monte 
Carlo  simulation  at  a  much  reduced 
computational  burden.  Figure  9  shows  the 
motion  prediction  of  TACTIC.  In  this 
simulation,  a  dynamic  OOI  starts  at  the  far 
right.  Based  on  its  history,  we  assume  that  it  is 
headed  to  the  yellow  line  to  the  far  left.  The 
map  has  a  series  of  mobility  obstacles.  Green 
areas  represent  predicted  high  probability  areas, 
while  red  areas  provide  low  probability  areas. 


Figure  9  Mission  module  predicting  the  motion 
of  a  dynamic  Object  of  Interest  around  a 
building. 
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In  this  case,  the  OOI  is  likely  to  walk  north  or 
south  of  the  obstacle;  however,  the  OOI  is  not 
likely  to  walk  through  it.  This  prediction  is  a 
probability  density  function  (PDF)  in  (x,y,t). 
MM  generates  and  maintains  these  PDFs. 
Based  on  the  PDFs,  and  using  the  areas  that 
have  been  previously  explored,  the  MM 
provides  information  about  the  likelihood  of 
finding  a  particular  OOI  in  a  particular  location 
at  a  particular  moment  in  time  and,  most 
importantly,  the  probability  of  entering  in  a 
death  zone  created  by  a  dynamic  OOI  at  a 
moment  in  time.  By  utilizing  the  GMM  and  by 
predicting  the  vehicle’s  own  velocity,  the 
planners  at  the  coordination  layer  can  plan  to 
rendezvous  with  OOIs  to  initialize  the 
neutralization  procedures.  At  the  autonomous 
mobility  layer,  this  cost  discourages  entering 
rooms  (without  first  clearing  the  entrance)  as 
well  as  cutting  corners  around  the  buildings  too 
sharply  before  first  clearing  the  areas.  It  also 
provides  guidance  during  the  neutralization 
procedure  so  that  a  set  of  collaborating  sensor 
UGVs  are  not  cornered  into  situations  that 
would  force  them  to  enter  the  kill  zone  of  the 
dynamic  OOI. 


wheel  encoder  MEMsIMU  GPS 


_  ,  Visual  odometry 

LADAR  odometry 

Figure  10  Pose  estimation  takes  under 
consideration  wheel  encoders,  MEMS  IMU, 
GPS,  LADAR  odometry  and  visual  odometry. 


5.  Human-machine  interface 
(HMI) 


4.  Operations  in  GPS-denied 
environments 

The  onboard  navigation  system  is  responsible 
for  determining  the  pose  of  each  robot,  both  the 
position  (x,  y,  z)  and  orientation  (roll,  pitch, 
yaw).  This  system  must  work  in  buildings,  near 
obstructions,  and  in  the  open. 

Our  team  has  extensive  experience  developing 
and  using  navigation  systems  for  both  robots 
and  people.  For  the  UGV  we  enhanced  our 
existing  adaptive  Kalman  filter  with  inputs  from 
a  6  DOF  MEMS  IMU,  wheel  encoders, 
Differential  GPS,  visual  odometry,  and 
LADAR  based  map  registration.  See  Figure  10. 
The  result  is  a  system  that  performs  well 
indoors  and  outside  and  especially  ensures  a 
smooth  transition  between  GPS  and  non-GPS 
environments. 


5.1  Realtime  Interaction 

The  Human  Machine  Interface  (HMI)  monitors 
and  controls  the  three  different  levels  of  the 
system  controller:  the  Platform,  Autonomous 
Mobility,  and  the  Coordination  levels.  The 
software  is  designed  to  be  operated  from  a 
touch  screen  or  from  a  more  standard  mouse 
and  keyboard  combination.  The  layout  and 
organization  is  based  on  previous  HMI  designs 
and  borrows  aspects  from  online  team  video 
game  strategies.  See  Figure  11. 

The  Coordination  Layer  HMI  has  been 
designed  to  aid  the  operator  in  viewing  and 
modifying  the  coordination  level  plans.  At  this 
level  the  operator  is  not  required  to  interact  with 
individual  trajectories  of  vehicles,  but  rather 
with  coarse  tasks  and  coarse  scheduling 
decisions. 

The  Autonomous  mobility  HMI  utilizes 
existing  control  methods:  teleoperation,  drive 
by  waypoints,  movable  waypoints.  RR  and  GD 
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Figure  11.  The  HMI  has  been  designed  to  provide  efficient  control  from  a  keyboard  or  from  a  touch  panel  It 
provides  maps,  area  coverage,  coordination  and  vehicle  status. 
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previously  developed  and  integrated  these 
methods  under  the  VTI  program  (VTI,  POC 
Jillyn  Alban  TARDEC).  The  Neutralization 
HMIs  provides  two  modalities.  To  neutralize  a 
moving  OOI,  two  video  streams  are  shown  to 
the  operator  for  each  UGV  involved  with  the 
neutralization  procedure.  Results  of  dynamic 
OOI  detection  are  framed  so  as  to  minimize 
operator  burden. 

The  Platform  HMI  provides  status  of  the 
different  mechanical  aspects  of  the  UGVs  and 
allows  the  operator  to  reboot  subcomponents. 
This  status  is  hierarchical,  and  additional 
information  can  be  gleaned  simply  by  touching 
the  appropriate  status  symbol.  Additionally, 
with  warning  and  error  states,  commonly 
recommended  operator  actions  are  presented  in 
a  natural,  seamless  fashion. 


map.  As  the  robot  traverses  an  area,  the 
onboard  cameras  record  the  surroundings.  The 
result  is  an  enhanced  viewing  of  the  mission. 
The  operator  can  change  the  pan,  tilt,  and  zoom 
of  the  image  and  the  software  will  generate  the 
desired  view  from  the  omni-directional  images 
collected  onboard.  It  also  displays  the 
navigation  solution  from  the  vehicle  as  well  as 
an  optional  2D  or  3D  LADAR  map  display. 
See  Figure  12. 

6.  Team  Breakdown 

Robotic  Research,  LLC  -  System  integration, 
hardware  and  software  design,  navigation, 
video  processing,  autonomous  mobility,  multi¬ 
robot  coordination,  operator  control  station, 
testing,  and  configuration  management. 


5.2  After  Action  Capabilities 

An  After  Action  Review  (AAR)  toolkit  was 
developed  to  provide  the  operator  with 
information  that  he/she  was  not  able  to  capture 
in  real-time.  RR-Viewer  displays  a  virtual 
camera  image  geolocated  with  the  generated 


Del  Services  -  system  integration,  LADAR 
perception,  autonomous  mobility,  and  testing. 

QinetiQ-NA  (Parent  company  of  Foster  Miller 
and  Applied  Perception)  -  base  platform, 
control  and  Symphony  interface,  shipping, 
Talon  support  in  Australia. 
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Figure  12.  RR-Viewer  allows  the  operator  to  pan,  tilt,  zoom  and  teleport  to  navigate  the  scene.  2D  and  3  D  maps 
are  available  as  well  image  recall  and  playback  from  prior  locations  similar  to  Google ’s  Streetview  system. 


Hhu*  CttiUMa 
JP  yi* 

Md  Hm  Nmi 

i-sKs  n-i »»? 


General  Dynamics  Robotic  Systems 

enclosure  design,  part  fabrication,  business 
development  support. 


Cedar  Creek  Defense  -  communications. 


Embry-Riddle  Aeronautical  University 
(ERAU)  -  hardware  design  trades,  laser  pointer 
mount  design,  system  assembly,  testing.  Two 
ERAU  interns  worked  at  Robotic  Research 
during  the  summer  of  2010. 


7.  Summary 

The  RASR  entry  to  the  MAGIC  2010 
competition  provides  robust  autonomous 
mobility  for  small  UGVs  as  well  as  an 
innovative  coordination  strategy  capable  of 
dealing  with  communication  losses.  The  team 
took  on  the  challenge  of  using  a  militarily 
relevant  platform  and  a  minimum  set  of 
relatively  inexpensive  navigation  and  LADAR 
sensors  because  this  is  the  most  likely  pathway 
to  deployment  for  the  near-term  benefit  of  the 
soldier. 


